Models as Approximations, Part I: A Conspiracy of Nonlinearity and Random Regressors in Linear Regression

Abstract. More than thirty years ago Halbert White inaugurated a “modelrobust” form of statistical inference based on the “sandwich estimator” of standard error. This estimator is known to be “heteroskedasticityconsistent”, but it is less well-known to be “nonlinearity-consistent” as well. Nonlinearity, however, raises fundamental issues because regressors are no longer ancillary, hence can’t be treated as fixed. The consequences are severe: (1) the regressor distribution affects the slope parameters, and (2) randomness of the regressors conspires with the nonlinearity to create sampling variability in slope estimates — even in the complete absence of error. For these observations to make sense it is necessary to re-interpret population slopes and view them not as parameters in a generative model but as statistical functionals associated with OLS fitting as it applies to largely arbitrary joint x-y distributions. In such a “model-robust” approach to linear regression, the meaning of slope parameters needs to be rethought and inference needs to be based on model-robust standard errors that can be estimated with sandwich plug-in estimators or with the x-y bootstrap. Theoretically, model-robust and model-trusting standard errors can deviate by arbitrary magnitudes either way. In practice, a diagnostic test can be used to detect significant deviations on a per-slope basis.